Managing test data in large scale performance environment

ABSTRACT

A method of processing a database can include comparing, using a processor, a delta file with a risk assessment criterion, wherein the delta file is generated from a first schema and a second and different schema, assigning a risk level to a change specified within the delta file according to the comparing, and applying the change of the delta file to a test database conforming to the first schema according to the assigned risk level.

BACKGROUND

One or more embodiments disclosed within this specification relate toprocessing a database.

Large-scale performance systems typically utilize a database that storesthe data used or referenced by the system. In order to test such asystem, a large amount of test data must be generated and populatedwithin a test database that can be used by the system. A test databasethat includes approximately one million or more records is not uncommon.Generating the test database that includes the records and allassociated metadata can take hours to complete. In some cases,generation of the complete test database can take more thanapproximately 24 hours.

When attempting to correct a defect or when updating the system, themodification that is implemented can require a change to the schema ofthe database. Testing the modified system necessitates the generation ofa new test database that conforms to the new schema. This process, likegenerating the original test database, can take a significant amount oftime to complete, thereby delaying testing of the modified system andrelease of the correction or update to the system.

BRIEF SUMMARY

One or more embodiments disclosed within this specification relate toprocessing a database.

An embodiment can include a method of processing a database. The methodcan include generating a delta file from a first schema and a second anddifferent schema, applying the delta file, using a processor, to thedatabase, wherein the database conforms to the first schema, andtranslating data of the database in accordance with a metadata file.

Another embodiment can include a method of processing a database. Themethod can include comparing, using a processor, a delta file with arisk assessment criterion, wherein the delta file is generated from afirst schema and a second and different schema, and assigning a risklevel to a change specified within the delta file according to thecomparing.

Another embodiment can include a system for processing a database. Thesystem can include a processor. The processor can be configured toinitiate executable operations and/or functions as described within thisspecification.

Another embodiment can include a computer program product for processinga database. The computer program product can include a computer readablestorage medium having stored thereon program code that, when executed,configures a processor to perform the various functions and/oroperations described within this specification.

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS

FIG. 1 is a block diagram illustrating an example of a data processingsystem configured for database processing in accordance with anembodiment disclosed within this specification.

FIG. 2 is a block diagram illustrating a data processing systemconfigured for database processing in accordance with another embodimentdisclosed within this specification.

FIG. 3 is a flow chart illustrating a method of processing a database inaccordance with another embodiment disclosed within this specification.

FIG. 4 is a flow chart illustrating a method of processing a database inaccordance with another embodiment disclosed within this specification.

FIGS. 5A and 5B, taken collectively, form a flow chart illustrating amethod of processing a database in accordance with another embodimentdisclosed within this specification.

DETAILED DESCRIPTION

As will be appreciated by one skilled in the art, aspects of the presentinvention may be embodied as a system, method or computer programproduct. Accordingly, aspects of the present invention may take the formof an entirely hardware embodiment, an entirely software embodiment(including firmware, resident software, micro-code, etc.) or anembodiment combining software and hardware aspects that may allgenerally be referred to herein as a “circuit,” “module” or “system.”Furthermore, aspects of the present invention may take the form of acomputer program product embodied in one or more computer readablemedium(s) having computer readable program code embodied, e.g., stored,thereon.

Any combination of one or more computer readable medium(s) may beutilized. The computer readable medium may be a computer readable signalmedium or a computer readable storage medium. A computer readablestorage medium may be, for example, but not limited to, an electronic,magnetic, optical, electromagnetic, infrared, or semiconductor system,apparatus, or device, or any suitable combination of the foregoing. Morespecific examples (a non-exhaustive list) of the computer readablestorage medium would include the following: an electrical connectionhaving one or more wires, a portable computer diskette, a hard diskdrive (HDD), a solid state drive (SSD), a random access memory (RAM), aread-only memory (ROM), an erasable programmable read-only memory (EPROMor Flash memory), an optical fiber, a portable compact disc read-onlymemory (CD-ROM), a digital versatile disc (DVD), an optical storagedevice, a magnetic storage device, or any suitable combination of theforegoing. In the context of this document, a computer readable storagemedium may be any tangible medium that can contain, or store a programfor use by or in connection with an instruction execution system,apparatus, or device.

A computer readable signal medium may include a propagated data signalwith computer readable program code embodied therein, for example, inbaseband or as part of a carrier wave. Such a propagated signal may takeany of a variety of forms, including, but not limited to,electro-magnetic, optical, or any suitable combination thereof. Acomputer readable signal medium may be any computer readable medium thatis not a computer readable storage medium and that can communicate,propagate, or transport a program for use by or in connection with aninstruction execution system, apparatus, or device.

Program code embodied on a computer readable medium may be transmittedusing any appropriate medium, including but not limited to wireless,wireline, optical fiber, cable, RF, etc., or any suitable combination ofthe foregoing. Computer program code for carrying out operations foraspects of the present invention may be written in any combination ofone or more programming languages, including an object orientedprogramming language such as Java™, Smalltalk, C++ or the like andconventional procedural programming languages, such as the “C”programming language or similar programming languages. The program codemay execute entirely on the user's computer, partly on the user'scomputer, as a stand-alone software package, partly on the user'scomputer and partly on a remote computer, or entirely on the remotecomputer or server. In the latter scenario, the remote computer may beconnected to the user's computer through any type of network, includinga local area network (LAN) or a wide area network (WAN), or theconnection may be made to an external computer (for example, through theInternet using an Internet Service Provider).

Aspects of the present invention are described below with reference toflowchart illustrations and/or block diagrams of methods, apparatus(systems), and computer program products according to embodiments of theinvention. It will be understood that each block of the flowchartillustrations and/or block diagrams, and combinations of blocks in theflowchart illustrations and/or block diagrams, can be implemented bycomputer program instructions. These computer program instructions maybe provided to a processor of a general purpose computer, specialpurpose computer, or other programmable data processing apparatus toproduce a machine, such that the instructions, which execute via theprocessor of the computer, other programmable data processing apparatus,or other devices create means for implementing the functions/actsspecified in the flowchart and/or block diagram block or blocks.

These computer program instructions may also be stored in a computerreadable medium that can direct a computer, other programmable dataprocessing apparatus, or other devices to function in a particularmanner, such that the instructions stored in the computer readablemedium produce an article of manufacture including instructions whichimplement the function/act specified in the flowchart and/or blockdiagram block or blocks.

The computer program instructions may also be loaded onto a computer,other programmable data processing apparatus, or other devices to causea series of operational steps to be performed on the computer, otherprogrammable apparatus or other devices to produce a computerimplemented process such that the instructions which execute on thecomputer or other programmable apparatus provide processes forimplementing the functions/acts specified in the flowchart and/or blockdiagram block or blocks.

One or more embodiments disclosed within this specification relate toprocessing a database and, more particularly, to managing test dataacross multiple builds of a system. In accordance with the inventivearrangements disclosed within this specification, a database that hasbeen generated for use with a first version (e.g., build) of a systemcan be updated, or transformed, for use with a second and differentversion of the system. The processing can include, for example,transitioning the database from conforming to a first schema utilized bythe first version of the system to conforming to a second schema that isdifferent from the first schema. The second schema can be one that isutilized by the second and different version of the system. The datawithin the database also can be translated, when applicable, using oneor more user directives. The data, for example, can be translated from afirst data format used by the first version of the system to a seconddata format that is different from the first data format and which isused by the second version of the system.

In a further aspect, the various changes required to transition thedatabase from conformance with the first schema to conformance with thesecond schema can be evaluated for risk. For example, a level of riskcan be determined and assigned to each change needed to transition thedatabase to conform to the second schema. The risk of introducing errorsor otherwise corrupting the database with each change that is requiredcan be determined. Changes can be automatically applied, appliedselectively, or not at all based upon the determined risk of eachrespective change. Further aspects of the embodiments will be describedin greater detail within this specification.

FIG. 1 is a block diagram illustrating an example of a data processingsystem 105 configured for database processing in accordance with anembodiment disclosed within this specification. As shown, dataprocessing system 105 can be communicatively linked with a source coderepository (repository) 110, e.g., another data processing system ordata storage system. Repository 110 can store a plurality of differentversions (builds) of a system that is under development or under test.The system versions stored within repository 110, for example, can beversions of a software-based system that utilize or execute inconjunction with a database. The system versions stored withinrepository 110, for example, can be a large scale, performance-basedsystem that executes in conjunction, or cooperation, with a database.Repository 110 can store the source code for each of a plurality ofdifferent versions of the system. For purposes of reference andillustration, the system versions stored in repository 110 can bereferred to as system version “N,” where N can be a numeric valuespecifying a version number of the system. From time to time, systemsstored within repository 110 also can be referred to as the “systemunder test” or a “test system.” The version number may or may not beused when referring to a system under test or a test system.

For purposes of illustration, a first version of the test system (testsystem 1) utilizes a database defined by schema 115. A second anddifferent version of the test system (test system 2) utilizes adifferent version of the database defined by schema 120. Schema 120 isdifferent from schema 115. Each schema generally specifies thestructure, e.g., the hierarchy, of the database utilized by eachrespective test system. A schema, for example, can specify the variousobjects of the database, e.g., tables, columns, rows, properties orrequirements for fields in terms of data formatting and/or allowabledata types, etc. Test system 2 can incorporate a fix for a particulardefect found in test system 1. The resulting modification to test system1 results in test system 2 and introduces a change into schema 115resulting in schema 120.

FIG. 1 further illustrates that a database 125 has been created.Database 125 has a structure that conforms to schema 115 and can be usedby test system 1. Further, the data stored in database 125 conforms to afirst data format that is usable by test system 1. In one example,database 125 can be a test database that is created for purposes oftesting test system 1. Because test system 2 utilizes a database thatconforms to schema 120 (and likely has a different data format, e.g., asecond data format), database 125 is unusable with test system 2,whether for actual use, testing, or the like.

For example, testing, often referred to as a “test cycle,” can includecomponent verification testing (CVT), system verification testing (SVT),and regression testing. Testing further can include tests for stress,reliability, scalability, security breaches, and performance. During theperformance phase of SVT, the test environment can be loaded with asmany objects as are supported by the test system and then run understress conditions to induce problems that would only manifest duringlong run testing. After the test environment is loaded with objects forperformance testing, a backup of the test environment can be created sothat if or when a defect is found and fixes are placed in the testsystem, the test environment can be restored quickly and efficiently sothe fixes to the test system can be verified.

Consider the case in which database 125 is a test database (e.g., a testenvironment) that is generated to include a million or more unique keysand accompanying metadata. Generating such a test database can take asignificant amount of time. Further, consider the case in which a defectin test system 1 is discovered through testing conducted using database125. Test system 2 is created as a consequence of modifying test system1 to correct the defect. In accordance with the embodiments disclosedwithin this specification, database 125 can be transformed for use withtest system 2. For example, database 125 can be transitioned to conformto schema 120 for use in testing test system 2 as opposed to generatingan entirely new test database that is compliant with schema 120.Further, data that is formatted in the first format corresponding totest system 1 can be translated into a second and different format thatis utilized by test system 2. Transforming database 125, as opposed togenerating an entirely new test database which can require time periodsof approximately 24 hours or more in some cases, can significantlyreduce the time required for testing test system 2.

Accordingly, in one aspect, data processing system 105 can be configuredto obtain schema 115 and schema 120 from repository 110. Data processingsystem 105 can compare schema 115 with schema 120 and generate a deltafile 130. Delta file 130 can specify each of the differences, e.g., eachof the changes, determined between schema 115 and schema 120 during thecomparison. For example, schema 115 and schema 120 can be specified inany of a variety of different formats including, but not limited to, amarkup language file (eXtensible Markup Language), data descriptionlanguage, or the like. Similarly, delta file 130 can be implemented orspecified using the same or similar conventions. In general, delta file130 can specify structural changes to be applied to database 125. Forinstance, delta file 130 can specify new objects to be created, e.g.,new tables, changes to existing objects, e.g., new columns, or the like.

FIG. 2 is a block diagram illustrating a data processing systemconfigured for database processing in accordance with another embodimentdisclosed within this specification. FIG. 2 illustrates an example ofdata processing system 105 as described with reference to FIG. 1. Likenumbers will be used to refer to the same items throughout thisspecification.

Data processing system 105 can include at least one processor 205coupled to memory elements 210 through a system bus 215 or othersuitable circuitry. As such, data processing system 105 can storeprogram code within memory elements 210. Processor 205 can execute theprogram code accessed from memory elements 210 via system bus 215. Inone aspect, for example, data processing system 105 can be implementedas a computer that is suitable for storing and/or executing programcode. It should be appreciated, however, that data processing system 105can be implemented in the form of any system including a processor andmemory that is capable of performing the functions and/or operationsdescribed within this specification.

Memory elements 210 can include one or more physical memory devices suchas, for example, local memory 220 and one or more bulk storage devices225. Local memory 220 refers to RAM or other non-persistent memorydevice(s) generally used during actual execution of the program code.Bulk storage device(s) 225 can be implemented as a hard drive or otherpersistent data storage device. System 105 also can include one or morecache memories (not shown) that provide temporary storage of at leastsome program code in order to reduce the number of times program codemust be retrieved from bulk storage device 225 during execution.

Input/output (I/O) devices such as a keyboard 230, a display 235, and apointing device 240 optionally can be coupled to data processing system105. The I/O devices can be coupled to system 105 either directly orthrough intervening I/O controllers. One or more network adapters 245also can be coupled to data processing system 105 to enable dataprocessing system 105 to become coupled to other systems, computersystems, remote printers, and/or remote storage devices throughintervening private or public networks. Modems, cable modems, andEthernet cards are examples of different types of network adapters 245that can be used with data processing system 105.

As pictured in FIG. 1, memory elements 210 can store a schema comparisonmodule 260 and a transformation module 265. Schema comparison module 260can be configured to compare a first schema with a second schema,determine or otherwise identify the differences between the two schemasand, thus, the changes to be applied to a first version of a databaseconforming to the first schema to generate a different version of thedatabase conforming to the second schema.

Transformation module 265 can be configured to implement the databaseprocessing that transitions a database from conforming to a first schema(e.g., schema 115) to one that conforms to a second and different schema(e.g., schema 120) based, at least in part, upon the delta filegenerated by schema comparison module 260. In one aspect, transformationmodule 265 can transition, e.g., modify, the database in place toconform to the second schema as opposed to creating a copy that isupdated. Transformation module 265 further can perform any datatranslations that may be required.

It should be appreciated that while schema comparison module 260 andtransformation module 265 are illustrated as being co-located withindata processing system 105, this need not be the case. For example, inanother aspect, a second and/or different data processing system can beused to generate a delta file from a comparison of the first schema andthe second schema. In that case, transformation module 265 can residewithin data processing system 105 without schema comparison module 260.Accordingly, data processing system 105 still can transform a databaseas described within this specification.

FIG. 3 is a flow chart illustrating a method 300 of processing adatabase in accordance with another embodiment disclosed within thisspecification. Method 300 can be performed by a data processing systemsuch as data processing system 105 of FIGS. 1 and 2.

In step 305, the data processing system can retrieve a first schema, forexample, from a repository. The first schema can define the database fortest system 1. In step 310, the data processing system can retrieve asecond schema from the repository. The second schema can define thedatabase for test system 2. In step 315, the data processing system cancompare the first schema with the second schema. In comparing the twoschemas, the system can determine each difference, e.g., each delta,between the two schemas.

In step 320, the data processing system can generate a delta file. Thedelta file can specify each change, or delta, that, if applied to adatabase that conforms to the first schema, results in the databaseconforming to the second schema. The differences determined in step 315,for example, can be specified within the delta file as one or morechanges that specify the particular actions to be taken to modify thedatabase to conform to the second schema.

FIG. 4 is a flow chart illustrating a method 400 of processing adatabase in accordance with another embodiment disclosed within thisspecification. Method 400 can be performed by a data processing systemsuch as data processing system 105 of FIGS. 1 and/or 2. Method 400 canbegin in a state in which a database that conforms to the first schemahas been generated. For example, the database can be a test databasegenerated for testing test system 1. The database can be archived orotherwise stored for subsequent use.

In step 405, the data processing system can retrieve the archived (orstored) database that conforms to the first schema. For example, thedatabase can be archived in a compressed format that retains thestructure of the database, e.g., the directory tree of the databaseand/or file structure. While a database can be archived using the JARfile format, the embodiments disclosed within this specification are notintended to be limited to any particular technique for archiving orstoring a database. For example, any of a variety of differenttechnological solutions are available for archiving or backing up testenvironments and can be used with the embodiments disclosed within thisspecification.

In step 410, the data processing system can restore the archiveddatabase to a target data processing system (e.g., within the dataprocessing system that is performing method 400 or to another dataprocessing system communicatively linked to the data processing systemimplementing method 400). For instance, the archived database can bedecompressed and stored on the target data processing system.

In step 415, the data processing system can apply the delta file to thedatabase. The data processing system can apply one or more changes, oreach change, specified by the delta file to the database. Examples ofchanges to the database that can be implemented can include, but are notlimited to, the addition of a column to a table, the addition of a newtable to the database, etc. After application of the delta file to thedatabase, the database conforms to the second schema, at least in termsof object structure and/or hierarchy. The data contained within thedatabase, however, is unchanged. The data within the database stilladheres to the first data format utilized by test system 1.

In step 420, the data processing system optionally can disable one ormore requirements for the database. The data processing system can, whenrequired, relax, e.g., disable, one or more requirements specified bythe first schema. For example, the database can include a column ofaccount numbers. One of the modifications to test system 1 can be amigration from the use of 12 digit account numbers to 15 digit accountnumbers. The first schema, therefore, specifies a requirement that eachentry in the account number column is to have 12 digits. The secondschema, however, can specify a requirement that each entry in theaccount number column is to have 15 digits.

To transform the database to specify 15 digit account numbers instead of12 digit account numbers, the data in the account number column must betranslated from 12 digit account numbers to 15 digit account numbers.Simply translating the data to 15 digit account numbers, however, willinduce an error since the resulting data in the account number columnviolates the requirement specified by the first schema. Thus, therequirement must be disabled.

Accordingly, in step 425, the data processing system can transform thedata within the database as may be required. In one aspect, a metadatafile 430 can be utilized to specify the particular transformations thatare to be applied to the data in the database. Continuing with theaccount number example, metadata file 430 can specify the manner inwhich the account numbers are to be transformed from 12 digit accountnumbers to 15 digit account numbers. Metadata file 430 can specify analgorithm to be used to update the account numbers to 15 digits. Inother example, e.g., where a new column and/or table is added, themetadata file can specify an algorithm indicating how to generate thedata that is used to populate the new column and/or table. For example,metadata file 430 can specify whether the data is to be randomlygenerated, is generated as a function of other data in the database,etc.

In step 435, the data processing system optionally can enable one ormore requirements for the database. In one aspect, requirements thatwere disabled can be enabled. In another aspect, requirements specifiedby the first schema that were disabled can remain disabled while one ormore other requirements specified by the second schema can be enabled,e.g., in lieu of enabling the requirements of the first schemapreviously disabled. Still, a combination of requirements for the firstschema and the second schema can be enabled. The example utilizingaccount numbers is one in which a requirement of the first schema(account numbers must have 12 digits) remains disabled while a newrequirement from the second schema (account numbers must have 15 digits)is enabled. In cases such as the aforementioned example, the newlyenabled or applied requirement can pertain to the same data that wastransformed and to which the relaxed requirement pertained. In anotherexample, however, both schemas can require that the account number beunique. Accordingly, this requirement is common to both the first schemaand the second schema. As such, when the requirement for unique accountnumbers is enabled, the data processing system is enabling a requirementof both the first schema and the second schema.

Method 400 illustrates a technique that can be used to transform adatabase to conform to a different schema. The transformation involvestransitioning the database from the first schema to the second schema.The transformation further involves translating data from a first dataformat corresponding to the first schema to a second and different dataformat corresponding to the second schema. Thus, rather than generatinga new database, e.g., from scratch, or copying the prior database andupdating the copy, the existing database can be updated or transformedin place.

FIGS. 5A and 5B, taken collectively, form a flow chart illustrating amethod 500 of processing a database in accordance with anotherembodiment disclosed within this specification. Method 500 illustratesan exemplary method of transforming a database in terms of both schemaand data formatting. Method 500 can be performed by a data processingsystem as described with reference to FIGS. 1-4 of this specification.Method 500 can begin in a state where a database to be transformed hasbeen created and restored, a delta file has been generated, and ametadata file specifying data format changes has been generated.

Method 500 illustrates an exemplary embodiment in which each of aplurality of changes specified in the delta file are assigned to one ofa plurality of risk levels according to a comparison of the change withrisk assessment criteria. Each of the changes can be implementedaccording to one of a plurality of different implementation procedurescorrelated with the risk level of the change as described in furtherdetail below. Further, users can be queried as to whether to performoperations when the operation exceeds a threshold, e.g., is assigned arisk level of medium or high. For example, responsive to determiningthat the risk level of the change is above a threshold risk level, thedata processing system can generate a prompt inquiring whether to applythe change.

Accordingly, method 500 can begin in step 505 where the data processingsystem can analyze the delta file and the metadata file. In one aspect,for example, the data processing system can search for instances ofkeywords within the delta file and/or the metadata file that indicateparticular actions such as addition of a table, addition of a column,deletion of a table, deletion of a column, the rendering of particulardata (e.g., a column) as obsolete, or the like.

In step 510, the data processing system can classify the pending schemachanges according to risk. For example, each of the changes specified bythe delta file can be assigned a particular risk level such as low,medium, or high. In illustration, risk assessment criteria can bespecified in which particular database operations can be correlated withkeywords that, when found within the delta file and/or the metadatafile, indicate that the correlated database operation must be performed.The database operation that are detectable and correlated with keywordseach can be enumerated and assigned to a particular risk level.

In classifying the changes needed to transform the database, the dataprocessing system can compare keywords located in the delta file and/ormetadata file with the risk assessment criteria. More particularly,keywords located in the delta file and/or the metadata file can bematched with keywords in the risk assessment criteria. The risk levelthat is associated with the matched keyword in the risk assessmentcriteria can be assigned to the change in the delta file and/or themetadata file. In one example, the risk levels can be defined as low,medium, or high. Low risk operations can include operations consideredunlikely to cause any corruption of the database. Medium risk operationscan include operations with an elevated risk of causing corruption ofthe database. High risk operations can include any operation for whichexplicit user approval is required for performing the operation. Furtherrisk level assessment criteria and examples will be described withinthis specification.

An example of a delta file is illustrated below after the deltas notedtherein have been classified in terms of risk. As such, the delta fileillustrates various operations. Within the example, the deltas arearranged according to risk level as determined by the data processingsystem and described within this specification. Further, within theexample, explanatory annotations of the operation to be performed areincluded in brackets [] in some cases. Risk level headings are alsoplaced in brackets. The portion in brackets is not part of the deltafile, but rather included for purposes of illustration.

  [Low Risk Changes] DROP TABLE KMT_OBSOLETE_TABLE DROP INDEXIDX_CERT_AUTOSTATE_CHANGE [Obsolete index] CREATE INDEXIDX_CERT_EXPIRTION_DATE ON KMT_CERT  (EXPIRATIONDATE) MINPCTUSED 20;[New Index] ALTER TABLE KMT_CERT ADD COLUMN ALIAS VARCHAR(100) NOT NULL; ALTER TABLE KMT_CERT ADD COLUMN KEYSTORE VARCHAR(100)  NOT NULL;ALTER TABLE KMT_CERT DROP CONSTRAINT PK_CERT; [Medium Risk Changes]UPDATE KMT__CERT SET STATE = STATE * 10 [Apply a function to an existing column] [High Risk Changes] ALTER TABLE KMT__CERT ADDCONSTRAINT PK_CERT PRIMARY KEY  (UUID, ALIAS, KEYSTORE); [Converting asingle column primary key to  multi column primary key] DROP INDEXIDX_CERT_HASHVALUE; [Dropping an existing index] CREATE UNIQUE INDEXIDX_CERT_HASHVALUE ON KMT_CERT  (HASHVALUE) MINPCTUSED 20; [Convertingan index to unique index]

In step 515, the data processing system optionally can prompt the userfor a decision. For example, the data processing system can present thepending changes that are to be applied to the database in conjunctionwith the assigned risk level of each respective change. In anotherexample, the data processing system can prompt the user whether toimplement a given change or continue to the next change prior toimplementing each change or prior to implementing each change of aparticular risk level. In still another example, the data processingsystem can prompt the user whether to implement changes of a particularrisk level or continue to changes of another risk level. For example,the data processing system can query the user prior to implementing lowrisk changes, prior to implementing medium risk changes, and again priorto implementing high risk changes. The embodiments disclosed within thisspecification, however, are not intended to be limited to the examplespresented.

In step 520, the data processing system can determine whether to proceedwith the database transformation based upon the response received fromthe user. Responsive to a user input indicating that the user wishes toterminate the database transformation, method 500 can end. Responsive toa user input indicating that the user wishes to continue with thedatabase transformation, method 500 can proceed to step 525.

In another aspect, the user can select only those deltas (andcorresponding operations) that are desired from a plurality of listeddeltas necessary to transform the database. When no operations areselected, method 500 can end. When one or more of the operations areselected, where each operation is associated with a risk level, method500 can continue processing only the selected operations (e.g., theselected deltas).

In step 525, the data processing system can load the database (e.g., thetest database). In step 530, the data processing system can determinewhether there are any low risk changes to be processed or applied to thedatabase. Responsive to determining that one or more low risk changesare to be applied to the database, method 500 can proceed to step 535.Responsive to determining that no further low risk changes remain to beapplied to the database, method 500 can proceed to step 545.

In step 535, the data processing system can apply the low risk change(s)from the delta file to the database. In one aspect, a low risk changecan be one that does not require any relaxation of a requirement inorder to apply the change and subsequently transform the data, ifnecessary. For example, the creation of a new table, the addition of anew column to a table, etc., can be considered low risk. After step 535,the database, at least with respect to the low risk change that wasapplied, conforms to the second schema. The data affected by the lowrisk change, however, still conforms to the first data format.

In step 540, the data processing system can translate the data for thelow risk change(s) applied in step 535. Transforming the data caninclude operations such applying an algorithm to existing data to derivenew data, generating new data from existing data in the database topopulate a column, table, or the like, and generating random data.Accordingly, data for any objects of the database that were modified instep 535 can be translated. After performance of step 540, the data thatis affected by the low risk change applied in step 535 is formatted inaccordance with the second data format used with the second schema.After step 540, method 500 can continue to step 545.

In step 545, the data processing system can determine whether there areany medium risk changes to be processed or applied to the database.Responsive to determining that one or more medium risk changes are to beapplied to the database, method 500 can proceed to step 550. Responsiveto determining that no medium risk changes remain to be applied to thedatabase, method 500 can proceed to step 575.

In step 550, the data processing system can disable schema requirementsfor the first schema. In one aspect, the data processing system candisable any schema requirements that relate to a particular medium riskdelta that is selected for processing and application to the database.In another aspect, any schema requirements for the first schema relatingto a medium risk change (e.g., all schema requirements relating to amedium risk change), or all changes regardless of risk level, can bedisabled. In still another aspect, the data processing system canperform an analysis to determine the particular requirements that mustbe disabled. For example, the data processing system can determine whichrequirements of the first schema will be violated by the translation ofdata to be performed for any medium risk changes to be applied asdetermined from the metadata file and the delta file. In that case, thedata processing system can disable only those requirements that need tobe disabled to avoid an illegal condition in which the translated datadoes not conform to the requirement of the first schema.

In step 555, the data processing system can save any changes to thefirst schema implemented within step 550 to a temporary file. Forexample, consider the case in which an account number column is 12digits in the first schema and is to be 15 digits in the second schema.Both schema require a unique account number. The requirement of thefirst schema, e.g., 12 digits and being unique, can be stored in thetemporary file. In step 560, the data processing system can translatethe data affected by the medium risk change being applied to thedatabase. For example, the 12 digit account numbers can be translatedinto 15 digit account numbers using the specified metadata.

In step 565, the data processing system can reconcile the differences inthe temporary file with the changes in the delta file. For example, thedata processing system can determine whether the old requirementsconflict with the new requirements. In this example, the 12 digitrequirement conflicts with the 15 digit requirement. The uniquenessrequired of both schemas does not conflict. In step 570, the dataprocessing system can enable the schema requirements, e.g., the rulesand constraints, based upon the reconciliation. Requirements that do notconflict (e.g., uniqueness of account numbers) can be enabled regardlessof the schema with which the requirement is associated. In the case of aconflicting requirement, only the requirement of the new, e.g., second,schema is enabled. Thus, referring to the prior example, the 15 digitrequirement is enabled whereas the 12 digit requirement conflicts, isdetermined to be obsolete due to the conflict with the requirement ofthe second schema, and is not enabled. Those requirements determined tobe obsolete can be discarded. Accordingly, the resulting requirementsthat are enabled can be a mixture of requirements from the first and thesecond schemas.

In step 575, the data processing system can determine whether there areany high risk changes to be processed or applied to the database.Responsive to determining that one or more high risk changes are to beapplied to the database, method 500 can proceed to step 580 in FIG. 5B.Responsive to determining that no high risk changes are to be applied tothe database, method 500 can proceed to step 592 in FIG. 5B.

In step 580, the data processing system can determine whether any of theremaining changes to be implemented include an enumerated type ofchange. Examples of enumerated types of changes can include, but are notlimited to, converting a non-unique index to a unique index in a table,adding multiple columns to the primary key, or the like. The enumeratedtypes of changes represent supported high risk operations that can beperformed in some situations. For example, the enumerated types of highrisk operations can be performed responsive to an affirmativedetermination in step 588.

In step 582, the data processing system can determine whether at leastone of the remaining changes, e.g., the high risk changes, include oneof the enumerated changes. Responsive to determining that one or more ofthe remaining changes do not include an enumerated change, method 500can proceed to step 584. Responsive to determining that one or more ofthe remaining changes do include an enumerated change, method 500 canproceed to step 586.

Examples of changes that are not supported and, as such, are notenumerated, can include dropping or removing a column or adding a columnthat cannot be computed from existing data and cannot be randomlygenerated, but still must conform to multilevel database constraints.For example, adding a new table with new primary and foreign keys is notsupported. Further examples of unsupported high risk operations caninclude moving a table from one tablespace to a larger tablespace.

In step 584, the data processing system can discontinue processing andoutput a message indicating why processing has terminated. For example,the message can indicate the type of changes that are left to beprocessed with the risk level associated with each respective change.The message can indicate that the type of high risk change is notsupported or considered too likely to result in corruption of thedatabase to be performed. The message also can indicate which changeshave been applied. After step 584, method 500 can continue to step 592,where the resulting database can be output. The resulting databaseincludes any low and medium risk changes that have been applied.

Continuing with step 586, the data processing system can determine theuniqueness of the columns of the database affected by the high riskchange to be implemented. The columns to be processed, for example, canbe scanned to ensure that the column(s) does not duplicate values. Instep 588, the data processing system can determine whether the column(s)satisfies the requirements in terms of uniqueness (e.g., does notduplicate values). Responsive to determining that the columns will notsatisfy the requirements, method 500 can proceed to step 584 andcontinue as described. In this case, however, the changes to thedatabase can include one or more high risk changes depending upon thedetermination in step 586. Responsive to determining that the columns dosatisfy the requirements, method 500 can proceed to step 590 where thehigh risk changes are applied to the database. It should be appreciatedthat numerous columns can be evaluated and that each column can beevaluated independently resulting in one or more high risk changes beingpotentially applied while one or more other high risk changes are not.

After step 590, method 500 can proceed to step 592, where the resultingdatabase is output. The resulting database, when proceeding to step 592from step 590 or step 575, will be a database that conforms to thesecond schema and that includes data formatted according to the seconddata format.

Method 500 is provided as an example of a database processing technique.In one or more other embodiments, various steps can be ordereddifferently. For example, in another aspect, high risk and/or mediumrisk changes can be evaluated prior to applying any change regardless ofrisk level to the database to ensure that the process can be completebefore making any changes. For example, steps 580-588 can be performedprior to other steps. Only responsive to determining that the high riskchanges can be applied will the data processing system first apply lowrisk changes, then medium risk changes, and then proceed with the highrisk changes.

In instances in which either the transition from a first schema to thesecond schema (application of a change) or a data translation fails, thedetails as to the changes that are being implemented can be saved andused to further enhance the decision making process in terms of what isconsidered low, medium, or high risk and whether the process shouldcontinue in view of any prior failures. For example, the data processingsystem can escalate the level of risk of a particular operationresponsive to that operation failing a designated number of times.

In another embodiment, a delta file can be evaluated for risk withoutcontinuing to implement the various deltas noted therein. For example,the delta generation functions can be executed in combination withoperations assigning risk to each of the deltas within the delta file.That information, e.g., the risk level of deltas, can be utilized bydevelopers to determine how a change to a schema will affect a largersystem under test and test data that has been generated for the systemunder test. Using such a tool, developers can determine whether toimplement a given change or defer the change to a later time.

The flowcharts and block diagrams in the Figures illustrate thearchitecture, functionality, and operation of possible implementationsof systems, methods and computer program products according to variousembodiments of the present invention. In this regard, each block in theflowcharts or block diagrams may represent a module, segment, or portionof code, which comprises one or more executable instructions forimplementing the specified logical function(s). It should also be notedthat, in some alternative implementations, the functions noted in theblock may occur out of the order noted in the figures. For example, twoblocks shown in succession may, in fact, be executed substantiallyconcurrently, or the blocks may sometimes be executed in the reverseorder, depending upon the functionality involved. It will also be notedthat each block of the block diagrams and/or flowchart illustration, andcombinations of blocks in the block diagrams and/or flowchartillustration, can be implemented by special purpose hardware-basedsystems that perform the specified functions or acts, or combinations ofspecial purpose hardware and computer instructions.

The terminology used herein is for the purpose of describing particularembodiments only and is not intended to be limiting of the invention. Asused herein, the singular forms “a,” “an,” and “the” are intended toinclude the plural forms as well, unless the context clearly indicatesotherwise. It will be further understood that the terms “includes,”“including,” “comprises,” and/or “comprising,” when used in thisspecification, specify the presence of stated features, integers, steps,operations, elements, and/or components, but do not preclude thepresence or addition of one or more other features, integers, steps,operations, elements, components, and/or groups thereof.

The terminology used herein is for the purpose of describing particularembodiments only and is not intended to be limiting of the invention. Asused herein, the singular forms “a,” “an,” and “the” are intended toinclude the plural forms as well, unless the context clearly indicatesotherwise. It will be further understood that the terms “include” and/or“including,” when used in this specification, specify the presence ofstated features, integers, steps, operations, elements, and/orcomponents, but do not preclude the presence or addition of one or moreother features, integers, steps, operations, elements, components,and/or groups thereof.

Reference throughout this specification to “one embodiment,” “anembodiment,” or similar language means that a particular feature,structure, or characteristic described in connection with the embodimentis included in at least one embodiment disclosed within thisspecification. Thus, appearances of the phrases “in one embodiment,” “inan embodiment,” and similar language throughout this specification may,but do not necessarily, all refer to the same embodiment.

The term “plurality,” as used herein, is defined as two or more thantwo. The term “another,” as used herein, is defined as at least a secondor more. The term “coupled,” as used herein, is defined as connected,whether directly without any intervening elements or indirectly with oneor more intervening elements, unless otherwise indicated. Two elementsalso can be coupled mechanically, electrically, or communicativelylinked through a communication channel, pathway, network, or system. Theterm “and/or” as used herein refers to and encompasses any and allpossible combinations of one or more of the associated listed items. Itwill also be understood that, although the terms first, second, etc. maybe used herein to describe various elements, these elements should notbe limited by these terms, as these terms are only used to distinguishone element from another unless stated otherwise or the contextindicates otherwise.

The term “if” may be construed to mean “when” or “upon” or “in responseto determining” or “in response to detecting,” depending on the context.Similarly, the phrase “if it is determined” or “if [a stated conditionor event] is detected” may be construed to mean “upon determining” or“in response to determining” or “upon detecting [the stated condition orevent]” or “in response to detecting [the stated condition or event],”depending on the context.

The corresponding structures, materials, acts, and equivalents of allmeans or step plus function elements in the claims below are intended toinclude any structure, material, or act for performing the function incombination with other claimed elements as specifically claimed. Thedescription of the embodiments disclosed within this specification havebeen presented for purposes of illustration and description, but are notintended to be exhaustive or limited to the form disclosed. Manymodifications and variations will be apparent to those of ordinary skillin the art without departing from the scope and spirit of theembodiments of the invention. The embodiments were chosen and describedin order to best explain the principles of the invention and thepractical application, and to enable others of ordinary skill in the artto understand the inventive arrangements for various embodiments withvarious modifications as are suited to the particular use contemplated.

1-25. (canceled)
 26. A computer-implemented method of processing adatabase conforming to a first schema, comprising: generating, using ahardware processor, a delta file based upon a comparison between thefirst schema and a second schema different than the first schema;updating, using the delta file, the database to conform to the secondschema; and translating data of the database in accordance with ametadata file, wherein the updating the database does not change thedata in the database.
 27. The method of claim 26, further comprisingassigning a change specified in the delta file to one of a plurality ofrisk levels according to a comparison of the change with a riskassessment criterion.
 28. The method of claim 27, wherein upon thechange being assigned a risk level below a threshold risk level, theupdating and the translating are performed without user intervention.29. The method of claim 27, further comprising generating, upon thechange being assigned a risk level above a threshold risk level, aprompt.
 30. The method of claim 26, wherein each of a plurality ofchanges specified in the delta file are assigned to one of a pluralityof risk levels according to a comparison of each change with riskassessment criteria; and each change within the delta file isimplemented according to one of a plurality of different implementationprocedures correlated with the risk level of the change.
 31. The methodof claim 26, wherein the metadata file specifies at least onetransformation for the data, during the translating, to conform the datawith the second schema.
 32. A computer hardware system configured toprocess a database conforming to a first schema, comprising: at leastone hardware processor configured to initiate the following executableoperations: generating a delta file based upon a comparison between thefirst schema and a second schema different than the first schema;updating, using the delta file, the database to conform to the secondschema; and translating data of the database in accordance with ametadata file, wherein the updating the database does not change thedata in the database.
 33. The system of claim 32, wherein the at leastone hardware processor is further configured to initiate assigning achange specified in the delta file to one of a plurality of risk levelsaccording to a comparison of the change with a risk assessmentcriterion.
 34. The system of claim 33, wherein upon the change beingassigned a risk level below a threshold risk level, the updating and thetranslating are performed without user intervention.
 35. The system ofclaim 33, wherein the at least one hardware processor is furtherconfigured to initiate generating, upon the change being assigned a risklevel above a threshold risk level, a prompt.
 36. The system of claim32, wherein each of a plurality of changes specified in the delta fileare assigned to one of a plurality of risk levels according to acomparison of each change with risk assessment criteria; and each changewithin the delta file is implemented according to one of a plurality ofdifferent implementation procedures correlated with the risk level ofthe change.
 37. The system of claim 32, wherein the metadata filespecifies at least one transformation for the data, during thetranslating, to conform the data with the second schema.
 38. A computerprogram product for processing a database conforming to a first schema,comprising: a hardware storage device having stored thereon program codethat, when executed, configures a hardware processor to perform:generating a delta file based upon a comparison between the first schemaand a second schema different than the first schema; updating, using thedelta file, the database to conform to the second schema; andtranslating data of the database in accordance with a metadata file,wherein the updating the database does not change the data in thedatabase, and the computer readable storage device does not consist of atransitory, propagating signal.
 39. The computer program product ofclaim 38, wherein the at least one hardware processor is furtherconfigured to perform assigning a change specified in the delta file toone of a plurality of risk levels according to a comparison of thechange with a risk assessment criterion.
 40. The computer programproduct of claim 39, wherein upon the change being assigned a risk levelbelow a threshold risk level, the updating and the translating areperformed without user intervention.
 41. The computer program product ofclaim 39, wherein the at least one hardware processor is furtherconfigured to perform generating, upon the change being assigned a risklevel above a threshold risk level, a prompt.
 42. The computer programproduct of claim 38, wherein each of a plurality of changes specified inthe delta file are assigned to one of a plurality of risk levelsaccording to a comparison of each change with risk assessment criteria;and each change within the delta file is implemented according to one ofa plurality of different implementation procedures correlated with therisk level of the change.
 43. The computer program product of claim 38,wherein the metadata file specifies at least one transformation for thedata, during the translating, to conform the data with the secondschema.